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ABSTRACT 

Examples from the evaluation of a prd^ram in which' 
data collection systems were developed jointly by the program's staff 
and evaluators are described. The Louisiana SPUR (Special Plan 
Upgrading cReading) Project was evaluated by the Louisiana Department 
of Education Bureau of Evaluation. SPUR involves 63 of the state's 66 
.public school systems and is implemented by a field-based s^taff of 
^approximately^ 60 technical assistants. The size and geographic 
dispersion of^the progifam ledl to several problems in data collection 
and evaluation usq. The SPUR ^ield staff; often acted as data 

. (jWl^ctors fo^* the evaluation^ a >rocedurjB that demanded careful 
^quality .asslirance'' of instr^uments and activities. With potential 
audiences ranging from state legislators to classroom teBchers, no 

. single report could meet all' needs for evaluative information. In the 
^first year of the evaluation , data, collection methods wefie develdped 
that, could be easily ,uS^d by SPUR field staff iand would have utility 
bdyond the evaluation itself . Exainple.s were developed cooperatively 
by- the evaluators and .pr5gram staff members. The instruments s^tye 
management dnd program improvement needs as well ag providing / 
evaluation information. Although the data^ collection , sys^bM^; 
described are - speci^^ic to the SPUR evaluation;,- their deviS^ipinent and 
use is appliCc^ble to other large scale , evaluations. ;(Author/DWH,) 
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DEVELOPMENT AKD USE OF' COOPET^ATJVE ctlENT-E VALUATOR 
DATA COLLECTION SYSTEMS - . 

• ABSTRACT . • 



t, (^;- 



This paper describes three examples from a 1a>ge-%ca1e eyalua^^^^^^^ 
which program st^iff were involved collaboratively in d^velop-ing 
evaluation, instruments and collecting data. The; exaj^le^;, include a 
iystem; for measuring student time on task, procedures for li 
staff field activiities » 'and the establishment of a , ^ ^ , ^ 
t4pn data-^ base. ^ The paper describes the manner i^ii"^^ 
collaboration was instituted and.manaQed, tfie. joint role's of evalua- 
program staffs, and the use^made of the data collection 
It ar:gues that the qual.ity pf evaluation Mnfbrmati on a the 



logging 
program parti cipa-^ 



. tion and 
systems. 

usefulness of evaluation findings can be increased by includihg 
program staff members as active, parti ciplwts iri an /^Valuation'. In 
several instances evaluation artivities'^hayie. beert incorporated by- the 
program as administrative or^ developmental ^procedures. • ' / 
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\ " . INTRODUCTION,/ • 

TMs paper describes three exampT.es from the evaluation of a 
single program in which^ata collection -systems- we jointly 
by the. program's -staff and evalcfator^r vT^ is the^ouisiana 

SPUR (Special Plan Upgrading Reading) Project, and it -is evaluated<.by 
the Louis'^ana Department of Education Bureau af Evaluation^ SPUR is a 
lar^e project, involving, 63 of :.the State's ,66 public schopl systems , 
and implemented by a field-based staff /of . approximatfwiy 60 .technical 
assistants.^ The size and geo^rapj(ic dispersion of the program'led to. 
'.several^'-pptential problems in -dat-a Vcqllection and evaluation use^. 
'First, the^PUR field staff acted in majtjiy instances as. ^data^^c^pl lectors 
for the evaluation, a procedure that demanded ^careful ^ qtl^j^l^^ assur- 
ance of instruments and activities. Second, with potentikl^' audiences 
ranging from State legislators- to cleissroom teachers jio,v|fhgle report 
^could me^t all needs for evaluative information. ' 

The approach taken in the first year of the evafluatibn was to 
develop data* collection methods that could be easily used by^ SPUR; 
field staff (who are reading experts,., not evaluatbrs) ' and that would 
have utility beyond the evaluation itself. ' The jthree examples shown 
here were developed cooperatively by the evaluators andvjthe program 
Staff menibers. They serve management and program improv^ernent needs as 
well al/ifJrol^O ding" eva^^ AUhough the data collec- 

, tion syisref^ described are specific to the SPUR evaluatHorf; the manner 
in' which they v/ere deyelbped and used' is applicable ^ to other large 
scale evaluations. 
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EXAMPLE 1: SPUR LOG SYSTEM 

- ■ ■ * . 



IT? 



This is a logging system developed in 1981-82 for the evaluation 
of the Louisiana SPUR (Special Plan Upgrading Reading) Project. The 
logging 'system documents J:he*ahiounts and kinds of services provided by 
SPUR personnels-technical assistants (TAS) and team ^leaders (TLs), 
The SPUR staff . and staff 'from the Bureau of Evaluation worked jointly 
to develop the logs ducing 1980-81 and' to revise the system for the 
1981-82 and 1^982-83 school years. The ^enti re SPUR staff , (persons 
housed in eigljt. regional offices) uses ^the. same procedures for logging 
'Staff tioje.^ 'JEachVteam member maintains a daily record that, when 
totaled, 'gives weekly servi,cte hours. The totals from the daily logs 
for eacK team member are transferred to one weekly supiary form that 
is sent monthly to ^ the State Departtnent of Educsition (SDE) for data, 
entry and anaTysiiSj^^jj^ ' . / ' , *. ? 

The two carlfe^c1>*it)n instrumentST^ the dally log, and the weekly 
summary log, are'^^esigned to record five principal* targets or five 
different kinds^ of , service in the l2^pecific service categories' 
.listed^, > / ' ' 



Kinds of Service 

out of) region 
2. genera] regional 
3^ maintai^ned - 
.4. demonstration sc\)dols* 
Si parishes 



Serviae Cateqor^4s^ 
Iv teaming / 
Z. ^preparation/management ^ 
3. . travel 

special assignments 
training: leadersMii 
tV*aining: teacher inservice 
training: parents and others, 
confer/ plan, centiral d(ffice 
t:onfer/plary, school staff 
confer/plan j other groups 
classroom assistance 



4. 
5. 
6. 

;7. 

8. 

JO*- 
11. 

12. 



Criteri^ of Excellence 
tommittees . . 
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An* logging SPUR service time is deciding , who 

are the targets . (who receives the services). After \the target 
determination is made, , the hoiirs of service are recorded under the 
appropriat6 service category heading, , The resulting logging' system, 
has, several -purposes: V ' . 

• ; prov'^s a ma^nagement t(^ol for the. SPUR staff " 
• ' • ^describes , and documents regional. SPUR activities, and ser- 
vices^ / 

/ • -allows decision makers to see how SPUR ijLSes time. and ^staff 
allocations ' " 

SPUR is divided into eight regipns. Each^region is unique in its 
organization and has different needs and requests, ; SPUR's goal was to 
develop a logging system that was comprehensive enougljiy to provide data 
comparable .across the eight SPUR regions, but would reflect the 
'.individual characteristics of each: 



ROLES * • ^ ; ^ 

The SPUR. staff consists of the primarily school -based TAIIs who 
serve more than one school system, and the regional TLs who' coordinate 
and direct regional activities. Although the SPUR te^ members are 
reported as a single group, and each completes the same SPUR log form, 
theserv^cqs of different team members vary* 

^In tha developmental phase the SPUR field staff members conveyed 
messages to the evaluators about "reality" problems, with the logging 
systen^. Suggestions and comments from the field staff were used in 
designing;the logs. The Bureau of Evaluation relied on the SPUR 
members to determine the' following: " ^ 

H what ki nds of iaformation^ they waVited the l ogs to^ gatheV- 
• what questions ^they Vfanted answered froih the logs , ; , - / 
^what comparisons they were interested iVi making from the "log 

whht problems: they had with the data collection instruments, 
and/or 'instructions for use 

What time lines were • required fqr data collection and/or 
data 

use • . ^ 



t 




Wlt^ these giiid<|^nes, §PUR fielfl^tafl^tidYtaff frbm the' Bureau 
if: Evaluation worked jointly , to^ ^eve for the SPUR 

program* * ' , - 

7 The Bureau of Evaluation trained' the eight regional TLs to make 
.sure that therer */as a uniform understanding of thfe system. Each It 
thenitrained hef regional field staff team members/ /t' 

. It;is the Bureau of Evaluation's responsibility to gather ffom 
the SPUR f ield staff suggestions and commients £m tjie logs each year so 
that' revisions can 'be tnade Vfor the' upcopmig year.^ The Bureau^of 
Evaluation incorporate^^' these revisicmsr intoVd working draft for 
spur's approval. Working drafts a>>e produced and disseminated to 
gattier changes and/or suggestion^^rior to printing. Fina.l drafts are 
produced only after , agrpemenjfe^s reached ^between SPUR and the Bureau 
of Evaluation. / ' . ^ ' ' 

Completed logs ore returned to the BureAu'of Evaluation for data 
entry and arialysis./ A SAS ('statistical analysis system) program was 
developed by the 5DE Bureau . of Management Information Systems to 
provide the analysis necessai^y to answer the, SPUR log questions. The 
^program, which . i^^dlpsigned^ to produce, mcjnthly and year-tordate totals, 
is updated ye'^rly to ^correspond to^the log revisions. ^ 

DUTIES ' ^ ' . ' k 



/ The dy^^ies of the SPUR' staff and the Bureau' of Evaluation are 
OUy^lined below: 

' ^ - SPUR ' 



1) develops, program questions to be^nswered from log^dat^^' 

2) conveys; .field staff problems of use ' • 

3) '^ makes /comments and suggestions for -improvements and revis- 
, - ions ' ^ ' ' ■ ■ ' '. ' ' 

4) /'meets with the Bureau of Evaluation to openly discuss the 

final drafts 

5) makes final reviews %nd coirjpents before printing , ^ 

6) trains team members with tKe Bure&u^of Evaluation 

7) uses tjie logs to document SPUR fielcl hours ^ • 

8) returns the completed weekly summaries on a monthly basis to 
the State Department of Education, bureau of Evaluation 



-9) 



... u 

' 2) 

• 3.). 

•6) 

9) . 
10) 



11) 
12.) 

13) 
14) 
15) 
16) 



•.uses the vinformation [irbvided by the Bureaif IKf Evaluatidri 
.'(log •data Information such as' fact shSetS:, repopts) 



gather 



. I - BuY^eau oT Ey.a1uation ^ 

meets with SPUR to definej^PUR' pi^ograrf/juest^id^ a|ji^! 
problems and/ or comments on use^'of present Tog 
(Jevqlops and revises f)resent ; log with ^ SPUR Input \ , ' 

distributes working log draft for ^PUR review 
meets with ^PUR to openly discuss and critique • 
revises arid develops final draft ^ • 
trains SPUR fi&ld staff ' * ' / ' . ' 

distributes instruments to SPUR team members for fieTd use 
receiA/^s instruments. from SPUR team members and prepares Vor 
data entry \ , ■ ^ * > ^ 

analyzes data with SAS prograrri ; ? * 

distributes monthly aqd year-to-dale loq reports to SPUR ^ . 
regions . T 
priepares draft or end-of-year log report 
distributes data , to SPUR SCfE project officers for- comments- 
and suggestions 

incorp^^rates revisions intd-draft C ^ / 

Prepar'e final reports ' , • ^ 

prepares fact sheets . . ^ , 

rerhains available at all times to discuss and'^nswer any 

field. questions, ' ' - ^ ^ 



HOW DEVELOP£D . - ^ . v - 

The 'SPUR field staff originally developed its own- logs' for 
in-house management and accqjjntabil ity purposes. These' five-page lags 
(Appendix '^) requested much narrative documentation and lengthy 
detailed responses. Daily, logging by the field staff members arid data 
summaVization by the SPUR project officers became so burdensome that 
%he information requested arid collected^as not used/ 

SPUR wanted ^lescriptive and accountability information regarding 
field activities. The or^iginal reporting system had i§ome of this 
information, but it was anecdotal 2^nd hot re'ported uniformly, across 
the regiiQns or school systems. Therefore, d^ta could not be . sorted or 
quantified. . 

The eveluatbr suggested a revised log (Appendix B), shorter and 
/Very structured,, to replace the original five-page, version^ This log 
was developed using the old .logs as gliides. Seveiral -categori^^ unc^er 
the old log headings wer'e grouped together to form the si6rvic^,..£3*e^br 
ies used for the new lo^'s. ' . . ' ' 



SPUIl^wdrteq directly wHh the Bureau of Evaluation to dekvelop^ajnd 
then'^ to revise the logs. , SPUR shares the responsibility for training 
SPUR field personnel in usfn^ t)]e Ic^gs'^after the initial trairiing is 
completed. , * ' \ *' * ^ 

' • > . . ; YEAR 1 ; 

■ V \ ' " ■ . . ' . • 

Theofifst year, 1980-8r,^,SPUR' decided 1:o use both sets of logs; 
A data cbTlectt|n system haid to be established so that each field 
Staff member could understand the definitions, the service categories, 
and the instructions for using the hew logs/. Therefore, training the_ 
SPUB field staff members had to be comprehensive enough to ensure a, 
uniform understanding of both logging systems. The. training consisted 
of detailed explanati.on, practice situations, and a' question and 4 
answer sessibn. Field staff member's then used the* logs and contacted 
the evaluetor if ther*e were any problem^ or questions during field 
use. I ' ^ ' . 

Usimg both Ibgs proved to be a ,duplication pf effort for both 

' ■ . ' ' ' ' ■„;,:'' ' ^ 

time and information! SpUR liked the shortpr logs, saw thiat they wfere 

workable, and decided to use-ijhly one log for the upcomin|j year. . . 
• . V. YEAR^2 



The SPUR field ataff members ^ssl'sted j'n Ibg reyisio^. Th^Jogs 
were Vevise'd with these, purpose* in 'mind: 



■ • - ' ■ . ■ ■ ■■■ ■ ■ ■ 

efficiency of completion> 'i ^ * 

■ t transmittal of needed data- without narrative requir*ements 
allowance for narrative explanation if desired 



• contputer storage of submitted data 



/ 



Year 2 revisions included the developm^nt^f two' separate logs (Appear 
dix C):^ the weekly' log for field staf^^'Uge- and the-supfiaVy Ipg that 
totals the weekly field staff logs. ('Detailed instructions were 



developed to supplement the revised logs. A summary from eaob 'region 
was submitted monthly *to the State Dejj^artment of Education. ^- Training 
for the second year was mi^h shorter since 6nly one log set was used 
and the fi^ld staff members had helped to* develop it. The training 



format was basically'.the same ^^s that used^the ^first -ytear. The M^^^^ 
of Evaluation conducted the trailing for the TLs and:i^roJec0Qfticers; 
Oihertfregional staff mfe^bers "were trained by their- TL.^ [ * • 

a!' . . . ' ; • - YEAR 3 \ ^ > 

■ ■ • ' . ,V': ■ • . / ■■^ 

Year 3 revisions included: ' . * • - 

■0 " 'developing two separate . logs (Appendix D):.a dail^ log, .to 
be used daily for one week by the field staff, an<y a weekly 
log,^which would total all daily logs "for a region ' 
t adding the target categories of "out af- region" and "main- 
tained schools" ^ . * ^ . 
' • deleting the service, categories of "ntimber contacts—profes- 
sional and "number^-'of contactS"Community" . 

• adtlihg the s'eMice of "criteria of excellence 
committees,^ an^"trairitng: parents" , ' 

' addijjg: the number of^' days worked and the number of TAs 

, workibg - • ' • > 

• ' changing the reporting. summary method to weekly logs j total, 

of ^al V field Ibgs, instead of all totals on dne monthly Ic^^^^ 

. - forpi ^ /'•■'.•■•;■" ■ ' ' ■ ''■ A ■ ' .'C ■ ■ 

" •; ^ revisTng- the instructions in cdnjunction :^With the. A^^^^^^ 
. changes (training for the-^third year was similar W;:jj^h^^^^^^ 
previous training sessions;) ' , . ^ . ; ^ '/^l:f^H':f^C:^:^;:^- 
' ■ . ■ ' > . ^' - ■ '''' 

, \ . Informatipn ll§e , . ' 

1, The log summaries provtde a continual record of the SPUR services 
received at three ' levels— regional system, and schoo\r-^by 

/pooling the efforts of all, field 'team members. 

2. . The log summaries provide 'a ha rgj copy means of loo|cing at hours 

of service by category and region. ^ r , 

* ■.* 
,3. SDE project of1\icefs and -TLs can determine what service categor- 
^ .ies need to be examined for excess time allotments. 

4;;^ The data allow the SPUR project . to demonstrate the services it 
' provides, and who receives i|;hese services, to planners and decis- 
• " \ ion makers. 
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' * ^-School Task Force ^ SSfSPUR Staff PC=Pte/Coimiunlty 



This, public. doctoent was^ubl^shed at a cost" 
of $ '.05 'p b ropy by the Printing Section; ' ' 
Department Edijcatlto, P. 0. Box 4^4064, Baton , 
Rouge, U, 70804, to perfotiD a function ^ the /,' 
Bureau of El^ntary Education, 'under authority^ 
of special exception by the Dlvjlaicin of Adninlii-' 
tration. This^ material waS prlntei^ in accordance 
; With! the standards' for printing by .state agAicies 
established pursuant to RiS.' 43:31. . \ 



ERIC / 



1/ 



12' 



Date 




Type of 
Service 


Eole 
Group^ 




site I 




Mmt M 

t— — • 




> 

.1 - 


* ■ ; 




■ ^ 






'•} • 






















1 ' _^ 


' ' ' ' •,' ■ . 










. I '1'. ■ ■ • , 


■ • 


















• 












J 


1 






















* 


1 


■ * 


j_ — 




> 






* 1 








> 


t 


















. » '1 » ■ 




t ' 

f 

: 










— « ■'' 


■ , ■ ■ 1 ■ 



This public docuot was pubiishecl at a cost .of : ' 
$ per dopy by the Printing Section, Dept. of - : 
EiicaQon, P,,0. BOK 4406(/ Baton Bouge, lAj; 70804, ; 
to perform a 'function of the Bureau of EkpMrti^ 
Eaucadoii; under authority of special excerttich by . 
the Division. of Adiiinistraticn. 'This inatmal was 
prirfed in accordance with the standi m print- 
ing 'by state agencies .established^ pursuant to R,S, 

4 3;31. ■ V ' ' 

I 'I . ■ • : 

I . ■ ■* ■ . .. 



*Ilus. is a sumnary of yser^ces and dpas not reflect the 
totai tine' spent by ttchnical assistants in the parish. 





SPUR-2 



Hegion: 



: ' Date? 



D^y Nutber!^ 



Woridfig Hours:__^ 
■ Travel Houib: 
Total Hcwrs!:" 



'■ * ' 

flctilvitiesv , 


Total, 
Nutfcer ' ' 


Hours 


' '' ^ \^ 

UXltlBiltS^ > 


"VISIWION 
baib StlK3ols 








,.teactibe Schools 




k 


V 


'SPURDirectots 








UATcdc Joroe . 








•Other ' 
















SPUR Director 




V 




•Gmtral OffiCB 
,'perscrtnel 








Principal* • 








, University 
. . .fiepresentative, . 


1 






'*Other,/ 
















, .iteadintj Conference 


' [ ^ 


\ 




Other Conferences' 


■ t 




« 


Workshops 
♦Other 








— — ^ 






.u. ' ' 



„Ma}pr AcocnpliEhinents:^ 



ERIC 



^ ;(specify). 



•'-• Activities 

ISriMGS 
T.A. Team 



MaT 



liEATaskPoroe 



SPUR directors 
Reqiotel SPUR Board 



jdyisbry/support Gtoup 
^Priixiipals ; 




ill 



lool Task Force 



, State'.T.A. br T. L, 
^ Project oi£[ceF 
iiator. 



Bvalual 



■'SQieF 



NAIffiGQiENT IXiriES 



m/mmm^mm 



V'l' 



Hours 



Gcnnehts 



\, 4" 



Ihis public, docuient was Ewlished at. Sioost pt J 
per copy ^ the Printing Secticn, bept, of Bclucation, 
P, 0, Ba>cv44064, Baton Rwge, IA,, 70804, to jperfoma , 
functicn of, the Bureau of Elfflientary Etotit^^^^ / 
authority of special exoeptim by the / ' : 

Aftdnistraticn. Biis watetlal was printed ih aooordanye 
with 'the standards for printing by state agencies | 
^jfltablished•pur8uant toR.S. 43;31, ' 



0 



SPUR TECHNfCAL ASSISTANT'S , \ (wblte copy - state off Ice cjj(y]i; ' 'M 



DAIinOG 



'Name;.. 



Date 



,. M TC W TH - F 
T'^ ', " 'fc^Clrde^ 



UoKk Sites: 



lotal No|cl<ig Tli^: 
Total TraVel Time: _ 
Total Hourp:^_ 



Code 



Leader 



Participant 



T 



Hou: 




■ ''Nmber of; 



LP 



*0 



Hat'erlalg DlsseBlnatlonV 



CKlassroom Demonstration O»0bsenatlon , C-Coriference H^Plannlng an^ Development DIS/DMlssaiiinatlon/ 
Dlstributiorf H-Presentatlon-Regional M^Presentatlon-Parlsh .M-Presentatlon-School, P-Other (specify In ^ 



-4 



comment section) , 
LP-Leadershlp Personnel SP«Supportive Personnel Haciilty Weacheri.Wther (specify) 
STP-School Task Force. SS-SPUR Staff PC^Parent/Coimnunity : > • - ' '„ ; ■ . 



Hiajor Accomplishments: 



This public document w^b published at a. cost of ^ • 
$ .pit p er copy by the Printing Section, Dept. ' \ 
of Education, P. O...Box 44064, Baton Roiige, LA, 
70804, to perform a function of the Bureau; of 
Elementary Education; under authority of special, 
exception by the Division of Administration. This , , 
material was printed In accordancei with the standards \" 
for printing by state agencies! established pursuant o-' 
t6 B s: 43!31 . ' 



■ 1:V 



ERIC 



Nane: 



Work' sites! l.^ 

■3. ' 



I. AcCMtles Code 



Leader 'j 



Participant 



CooBients/Follov-up: 



SPUR Technical Assistant's 
' Dally Log 



Day Numberj 



Date :' • 



5. 

6'. ' ^ 



Total Kdrklng Tine: 
Total Travel Tiine: _ 
Total Hours: 



SPiJR-l 



MTuWThP 
Cir&e 



Hours 



II. Personnel Code 
'Number of Participants; 



LP JP T- SS ' 0*> 



Materials 
Dlsseinlnatlon, 



(iajor Accomplishments: 



V 



ERIC: lisi 



Tills public document was puljllshed at a cost of' $ > . per copy 
by the Printing Sectlbn, Department of Education, P. 0. Box, 
Baton Rouge , LA, 7080A , to perform a function Of the 
Bureau of Elementary Education, under authority of special ex- 
ception, by the Division of Administration, This material was 
printed in accordance with the standards for' printing by state . 
agencies established pursuant to R.S, 43:31. , 



m5 



vSPURTECHNICaI ASSiSTANCtWEEKLy LOG 



NAHE: 



WEEK OF: 



to 



V Assigned Service Areajs); ; ' 



Regioh: 



Day 



(wii ite. copy state office copy)^ 



mit' 

SYSTEM/ 
SCHOOL 



TEAHING 



PREPARATION 
MANAGEMENT. 



C, 



c 

0 



c 

c c a 

C 0 Q 
H OJOO 



y a 

c 0) 

1$ 

0T3 

c V 0 
(D > u 



U1 



C ' 0 U 

•t- >, (I 

E rx 

a. 6 oc < I- o 



, SPECIAL 
ASSIGNMENTS. 



\ 



£■ 

a u 

(D ui n) 
c ^ - 

0 4) J 
O)*0< 

a: 



0) > 
0 *J 
u -.s - 
0 >u» C 
r L c 3 

0 4) Q) E 
(t) tf) u Ex 



J 0) 



0HFERENCIN6/ 
PLANNINb 



0 <o 
>- a 



U 0 iO 
4) c 0 0 0) 



CLASSROOH 
ASSISTANCE 



^ 0 

0', 

S 

CO 

^ u e 

(0 tl ' 

> 

V V 0 

«^ « 

(A C 



TOTALS 



ffl "2 
c 0 s >i 

• c 

0. 0**- 0 

• • u . • 
0 0(^00 

z, 'X .: z " 



r«NtH REPORTED:. 



■)' 



APPENDIX C ' 
• • . 1981-1982 V , 

SPUR WCNTHY TECHNICAL ASSISTANCE SUMMARY 




SERVICE TOr • 






• • • • 




^, • • 


■ ' # 4 » ^ 

General Reaionav 








' * : 


: : 


•, '•}■ • t . . 












• ' . • 
• * 


C : •-•"■■i:, v. /: ' ■ y 








\ ' \ 






• • • , • 

* • • . * • , , • 








\ * \' 




K : 


• • . . . f ' • . 


V / — ^ ' r 

/ ' - • \ ' • ! 


























• • ' ^ • • • * • 

• • • • • • 


■ 












j . • • • • • • . 

• • , • • • • ^ 

■ T — I ; 








: i 










•■J > I 










, .... 
. f • • • 
. ■ • - • 


* ' * " ', * * * 








• 


j: : 


■ ' • • • ' . • • : 














, • • • • • • • 

■ • • • ' • • 

: ! h ! i • ' 








: i 




■ V' 


: : 

: : : • • > 










: < : 




, . i; • 

• • • • ' I 

• . • • • • 

: : 














> . • • : : 

. . .• * ■ _ • 


— \ 






\ — 






: : . : 


' ^ ■ . 

at ^ \ \ 












: : : : i i ' 
: : :• i l ': 












• > • 


i : : i : : 


\ IX^ L- 




Y 


• • 




















I I . • • • 










f- — 




i i i : L ■ i ^ 
J : L^-^ ? i 














\ . ; I \ : i 

• , • ! • • • 



ERIC 



UPDATE AnAOittiin Ho i 

• ■ ^ 1* ? ' 

i|Attich an update If mm Fract1«t School^ liavi batn detlonatad* existing Pracl^lqe 
^Schools droppfdt nM^Cantral Offlct Task Forca mabtr(t} appointed. 



14 



^3 



REGIONT 



MONTH!. 



APPENDIX C 
1^81 - 1982 



SPUR 

MONTHLY 

TECHNICAL ASSISTANCE 
REPORT 



3 

.3 

no 



SERVICE TO: 



out OF REGION^ 
GENERAL REGIONAL 



B 

.3 

TO 
01 
-I 
Z CP 

If 

01 01 

<%a 

CD (T> 



J— 



V 



a CO 

a 

(5 o 
3 ^ 
<-♦■ 01 

v> — ^ 



O.OI 



n't to 



U3 



CD (0 

•n OJ 



H 



G ! 
=?! 

Ol I 

, 3^ If* O 

oiu i<n.3 
I n 

O 0» iO^ 

n- i .-ji-o 

3" CD 1^ — • 

CO 3 • o» 

-I IQ 3 

i ■ ' 
■ « - 



O-'O^Iri' o 
3-3 13-3 
O ICD -h 

0 CD i-5 CD 

11, 

c/> -o "O 
rt' lO 

01 01 IC 01 

-t> 3 ro 3 
I 



1^ 



I 
I 

I » 

I - 
I 
I 



I K 

I. 
I 
I 

I ' • 
I 

l> 

IV> o 
HA — ' 
1^ 01 

roi 5 
13 g 

iO.O 
ICD 3 
I 

, I . 



I 

is I 

13 • 

I 3 • 

• ! 

I CD f 

I CD • 

I I 



+— i- 



.1 ! ! 



I 
I 

I* 

I. 



^1— i- 



Ik- 
I 

JL 



-{ H 



H + 



-I— H- 



4— f 



■f 1 1- 



I 

-I . 
!• 
I 
I 

I-., 

I 

I 

I 
I 

I , 

I 

I 

I 
I 
I 

♦ 

I . 

. I ' 

I 
I 
I 

I 
1^ 
I 

I 
I 
I 

I . 

-J 

I 
I 

I 
I 
I 



I 

1^ 



I 

! ^ 



I 

•' 
I 
I 
I 
I 

I 

i» 
I 

I 
I 
I 



t. 

• 
I 

.1 
I 
I 
t 

.1 



MAINTAINED SCHOOLS 



tJML DAYS FOR «THE MONTH 



> TA>iIIs 



15 



.+ 1 TL 

' ■ ' ' I. 



TiAME: 



TA-I • TA-II . TL " WEEK OF: 
(circle) 



appeNddtd 

REGION: 



to 



SPUft TECHNICAL ASSISTANCE 
J DAILY. LOG •■ * . .. 

1982-1983 ' 



V ASSIGNED SERVICE AREASi: . 

NUMBER WORK DAYS IN MEEKr >. 
WORK DAY . throuflif 



i * i B 



SERVICE TO; 
SITE i 1 e 



is 



13 

to 




F ! 6 ! H 



!> ! 

IWI if- 



■15 
in 



'5 

-10* 



■ If* ' 
i(rt 
I 

i». 
la 

ia.T 

IV» fi* I' o» 

•2"^ ICD ICD lO^ 

lis 12 2. '.3 2. IS 2. 

a iiD a 
into 1*1 (p 



13 ^ 

ir^ Oi i^(0 



l(D 



HOf> 



! ^ 



SI 

i§2 

l«A Ot 

%o a 

l(D • 
I 
I 
I 

^ 



I 

it/io 
in o 

iff 



(A 

«A 

3 



r 



(A 
(/I 



isrsr 

l-tl 3 
l-<|» 



lo.r* I 

Irr O I 

la* a . I 

•TJtl I 

I • n . I 

I32l-ff 

ic Qi I a 

S3 I n 
• J' <P. 



I .1 

lO I 

r I 

im . . I . 
I I 

tn y ,i; 
19 I 



ir* 
ir* 

irt 
i(/i 

I 
I 



T1)E 



t 



WED 



THl^. 



SITE #.2 




MON 



I I I 



4 



TUE 



WED 



THU ' 



4W 



OTHER.SITES 



TOf 



HON 



4sr ! 



I ' I 
I I 

! ^! 



TUE 





HLU 1 1 1 1 II i 1 ! 

. 1 1 ; 1 1 1 v' 1 1 * i 

I 1 1 1 1 1 , 1 ' , 1 ! ' ! 

" TUII*' 1 k 1 , 1 • 1 , 1 • ,1 I ^ I ■ 

^ .jnU < 1 1 I I 1 1 ' 11 I ! 


— ' — i — ~» 

; 1 




— T w » 1.1 f i , i 1 1 i '! ■ 




■•- -r-^ — iT — . - 

GENERAL REGION ■ . - ' 


TOT 1 1 !. SI 1 • 1 ' I ! . i 1 






1 . I.' ^1 1^ 1 . iV^ 1 1^ ' ' 

u 1, 1 1 '^1 1 i> 1 1 






•Til 


V i 5 / I I— ; 1 ' ! - a t - 

E 1 ' S 1 l» 1 I.I I J ' I 






1 1 « 1 II 1 A 1 . 1 t : 

urn ' • • • . 1 1, • ♦ 1 ^1 , ^ • ! 
mtV 1 1 1 1 1 1 1 i ! !• 






THU {• i ' 


I I -1 w , 1 •■;.!■■!*.' 
1 , 1 1 I I ! " 1 
I I I 1 1 1 ! ! 


— ^-i 




'• fR 


J J''^ 1^ Lti. 


^ ' , » » r- 1 i j 

• ' .1 . 1 1 1 ,! 1 
1 1 1 1 . 1 ' \{ , j 


I 


■ OUT. 0^ REGIONT • - ? ' 


TOT 1 1 


i 1 ill' I 




■ HON, 


1 


■ . • .1 1 1 . 1 5 j 
\^^\^ ! ! ! : ! I 


■ 1 ^1 


■nr — ^ : — — 


— ~ — 

■TUE 




! ! .1 vi ! I 

1 1 I' 1 1.1 ! . ! 


- — 4—* 






— ! . > — 1 i ^ — ' 1 ^ i 1 1 ! 

I-T jn \ . ! ! - ! 








;.• J '/'^ 


i«- i r ^1 1 i i i 


— 1— ^ 








• * 

1 ' I' . i 




T/if' T If; I 


I .&tt ^^^-l I 1 1 


1 .■ ■ .1. 



\ 



PARISH 





■ N 

N 
1 

N 
\ 

N 

X 

■ 1 



% 



N 
t 




■ ■. SBOR LOG .. ■ . 

1983-81 SPUR YEAR-TO-DATE PARISH SUHHARY THROUGH ^U(IE 



REGION 1 



2 
0 

' .1 

3 
6 

1 

21 
.2 

87 

■7 

'H7 
3 



226 
16 

169 

24 

703 
50 

163 

' 20 

2lt1 
27 

239 
20 

638 
38 



89 

,6 

*1H1 
18 

150 

163 
20 

72 
8 

83 

,7 

378 
22 




9 
2 

27 
2 

629 
38 

2468 
1*2 



126 
29 

101 
8 

13 
1 1 

33 



. 73 
16 

128 
10 

379 
6 



12 
1 

11 
1 

2 

,0 

.' 7 
1 

' 17 
2 

12 
1 

208 « 
12 

33 

2 

3 

■•0 

5H 
6 

H3 
5 

0 

0 
0 

10 • 
1 

578 



5 



J51I* 53H2 23H9 ' 127i» 
16 2H , 10 6 



0 

' 0. 

2 
0 

7 

,1 

0 ' 

. 0 

. 26 
3 

•29 
2 

0 ' 
0 

18 

1,. 

It 
1 

I* 
0 

2 
0 

3 

' 0 
3 

, 1 

• 0 
0 

17 

1 ., 

3l|3 
6 

H58^ 
2 









• 


i. » 


121 


14 


• 85- 


144 


51 


9 ' 


1 


•6 


10 


^ ' 


25 


38 


105 


69 


19 




5 


14 


9 


2 


29 


i2 


74 


12* 


27' 


2 


0 


5 


1 




71 


'6 


145 


86 


25 ' 


9 


1 


17 


10 


3 


33- 


7 


-117 


127- 


12 




1 


13 , 


14 


' 1 


Itl 


28 


119 


87 


39 


3 


2 


10 


- 7 


3 


37 


B 


72 


168 


34 


2 . 


0 


4 


,10 


2, 


133 ' 

1 WW 


lit 


160 


185 


41 


10 


1 . 


Vl4^ 


14 


f 3 


97 


23- 


' 104 


' 54 


23 


. 12 


3 


13 


7 


3 


31* 


& 


43 


31 


' 168 


It 


0 


5 


»3' 


19- 


33 

WW 


7 


100 


*:274 


10 


l( 


1 


11 


% 30 


1 


10 


; '46 


,125 


, 103 


71 


1 


4 


11 


9 


* 6 


1»9 


2 


63 


25 


1 


11 


0 


. )«» 


6 


0 


< 38- 


19 


125 


132 


65 


3 


2 


10 


11 


5 


8 


0 


0 


0 


,'0 


0 


0 


t) 


0 


i 0 


12 


10 


40 


32 


i' 

13, 


0 


0 


1 


1 


1 



■ 

157 
20 

347 " 
25- 

155 
19 

219 

25 \ ■ 

355 
30 

83 
5 

439 

33 . 

333 
42 

158 

169 
19 

407 , 
37 

93 

473 
36 

fO 
0 

110 ' 
2 



771 224 'l497 '1529''! ^9 f 4074 



70 
5 

n : 

0 
0 

0 
0 

0 
0 

62 
7 

16 

1, 

20' 

.2'-'. 

32 

0 

0. 

^ 13 
1 

164 

;i7 
b 

0 

135 
11 

0 
0 

-183 
3 

750 
I 



317 
23 

334 
43 

•8^ 

,40 

334 
37 

409 
34 

1063 
63 

269 

<■ 20 

115 
15 

408 
45 



; 258 
. 26 , 

150 
14 

: 2^10 

256 
20 

; 1057 
64 

4792; 
•82 

11205 

. , 50 



1073 

: p. 

441 
57 

500 
36 

495, 
60 

.558; 
t 63' 

792' 
66 



/TOT 
HRS 



1390 

' . ' . ■ ' ■ 

77? 
1399 
629 

' 892 
1201 



626^ ^< 1689 

37''.''.' ■ :. 



1063. 
80 



673 
65 

492 
55 

651 
72 

949 

236 
53 

997 



603 
36 

1067 
16 



1332 
76^ 
^00 

'909. 

1099 



446 
1253" 
1660 



5879 ■' 



■ X 



50 



22441 



' r 



/ EXAMPLE SPUR LOG . / , , • . 
1963-8l» SPUR YEAfi-TO'OATE SUHHARY TpUOH JUHE ^ , . 
: • <' ;r /regional TOTALS' ' ' , I 



1 , »l 



TMU PRFP TBAUFI SPEC TRAIN' TRAIN' TRAIN: 'PLANi PLAHf PLAN: ClASS 'C OF E INDIRECT DIRECT, TOT 
^ HANACr a£ S TEACH* B oSlCE STAFF A HR8 

N 3,51^.5.3^2 2.3W 1.27^ W8 771 ' 22} J. W7 1,529 :'659. M7j; 750 11^205 , 11.236 .22.^ 

' jj M 2H 10 6 2 , ; 3 , 1 7 7 3 - ,, IB I 3 / 3" 



l^^^^^^^ ,508 1,32^ 999 . /Mg^ 



,116 



N 2.370 3.055 '^57 214 U71 106 1.216 1,566 W2^ U21 1,338 / , 8.231 V' W^ 

I :13:-.17 15 .1 , 1. 3 1. 7 9 Z 22 f, , w 

, , ^ • ' ' -'t' . ' \ ' ' ! ' .' ' . . 

:.'i;077 ;i.058 -; 808 J68 30p i79 . ;3; ;i 333 6^6 , 1« U03 ^7^ 2.9j[3" .3.387 ; 6.330 

\ 17 17 13 3 '5 3 ; 1 • : 5. 10, 2 21 . 3h 

V a^r M61 i,)69- ! Mi -M. - SM ;'io5,:'?» ,i2i9 « :,26J. . m;} W>j Mt,iK 

I '22 25 12 J 5 3 ,1 ' 8: 10; 1 ' ; ^ , ^ 

' ■ ' ' ' \\ , ^ , ' ' " • ' . ' '' '. < 

i,J5 a.M' ^'I ^Sj '^''21: 



(.59i21,i 18..72 3, W - ,4"^;«2f 1M07 Ml!! »,97T 56, J. IW.Ml 



N 16.599 Zl.SUO iK.Of-: J.Msi jvyij ^jo'i y,-'- 'vrj -' " it % 5 

ji , 15 20 , 12 , 3 . 3 ' .l| . 1 6 12 i '5 3 - V 



PARISH 



EXAMPLE SPUR LOG 



1^83-8l» SPUR YEAR-TO-OATE SUHHARYT^^^^ JUNE 
OUT OF REGION/ GENERAL REGIONAL, AND HAINTAINED SCHOOLS 
' : REGION V . ^ V v, 



TEAH PREP TRAVEL SPEC TRAIN: TRAIH: TRAIN; ^PLAN:' PUN: PtAN: ' CLASS C OF^E INDIRECT fiSfg 'JK' 
HaIaCE assign LEADER TEACH. PARENT OFFICE STAFF OTHER ASSIST COWiSV ASStST ASSIST HB$ 



.'HAINTAINED SCHLS. N 



.27: 101 128 '10^ 0 t : f 



2 



8 10 1 



0 3 



•20 



OUT OF REGION v'N 
GENERAL REGIONAL f( 

» ' ' ' 

llpGlONALtOTAL , H 



629 ' 13 l»15 578 *17 / 8 ' 0 
^38. 1 25 35 .1 0 0 



n 0 0 ■ 0 0 1057 603 1660. 

0 0 : 0 ;o ' 0 6lj W:^. , . . 



21,88, 1925 379 . 28!* ' 3W ^ 12 10 ;,. W : ,32 { 73 ! 110 . 183 l»792 1087 , 5879 
Ii2*/ 33' 6 ^ 5- ^ 0 0, 1 , 1 , Iv, . 2 3 82 . 10 , • 



'31W 2039 922 872 
36 23 • 10 .10* J» 



it 



58 ^9- 165 . I6l» 138 583" ' 318 
1 0 2 2 ^ 7 ' , l» 



6105> ■ 2687 8792 
69- 31 



0 • 



V 



I, 



•. 31 



32 
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HAT. IT IS ' ; , - , V/.' ■ , ■ . • ,■; 

. The SPUR (Special Plan, Upgrading Readlhg) Data Base s^stem' w&s 
developed for this reading instruction .iraprovemeTit project to collect 
^statistical information about the participants, the project status of 
the schools that are, involved, and the gr?ide levels , numbers of . 
teachers, and numbers of stubents who are affecteb by the SPUR effort. 
The computerized data base is maintained by the, project's e'&aluatprs . 
and used fpr reporting, project management, aiid ♦generating sampling 
lists^' The ijfiformation gathered "Includes the following variables: 

1. ^ Region Number: The participating public .school systenis are" 

grouped into eight' geographical regions across -the State; 
each of the eight is served by a regional SPUR team. 

2. jSchool Name: Self explanatory. . . • 

3. School. Code: The Louisiana Department of 'Education* assigns a 
six-digit' code to each public schpol in the State. Th.e 

/ first three, digits indicate school system^ the last three 

. digits indicate the school . ' This code allows the Data Base 
to be/ merged with other Sjtate informatim^ files. 

4. SPUR Proqram Status: The^fchools are iflentified as P (parti-: 
Dipating) or 0 i (other).'* A" part-jcipating school receives 

• \ SPUR technical ass^istajit ^eirvices on a regularly schecfuled 

basis. The other schools receive irregularly scheduled 
*prvices or are ihvolved in some ektrgir-school manner; such 
^ as through inservite training for alt teachers i>i a grade 
level vacrpss the ^otal scnool system. , 

. 5. Criteria of Excellence (COE) Status: The COE is a peer- 
review system for evaluating the total school reading 
program and recognizing areas of excellence.' Sc)io61s are 
identified as having met or not having met the COE. Those; 
. that have rtiet the* COE are further identified as- Model 

(outstanding in ail areas); Recognition (outstanding in some 
^ but not all areas); oryDisseminating'. (outstanding^ in "one 

' ' area but not ^meeting COt status in other areas). Schools 
. . the|t have met the COE at. somp point Jn the pastjMJt have not 
^ V pursued the re^evaluation necessary^ to maintain this'^status 

are identified as Former. . * \ 

6. Year^E First Attained: The last two digits of the calendar 
year in which the COE status was first met. 

* 21 



7. Year COE Last Attained: For: schools that have pursued j!: 
V" > re-evaluate, the last two digits of the year in which the- 
^ . . COE status was JQranted after re-eyaluation. Th.i 

. . blank for schools that have not bee^n re-evaluated. ; \ 

. ■ ■ ' ' ' ' " ' ' ■■ ■ ' . ' 

8. " Number of Teachersr Thevnumbeif* of teachers In the school- who" 

are affected by SPUR. SPUR works primarily with teacljers of 
• ^ kindergarten through eighth grade. 

• ■ ' ^: ^ : ■ \ . • ^ ■ ■. n'^ 

'9. fiumber of Students: The^umber of students in the classrooms 
„ of the participating tea,chers in the school. : 

la. Grades Affected:7A' three digit cpde; the first digit indi- 
cates the' lowest, grad<i in the school with which SPUR works 
/and t*ie last two' digits indicate the highest grade with. 
Jwhich SPUR works; f \ 

The SPUR Data Base is an evaluation data source'that has fepls^ced 
' an earl ier^ management acLiv.ity caVried out by the State SPUR project. 
The older format was a survey completed by thQ. regional team members 
^ - at the beginning of each -school -year that gave information aboui^ the' 
size of the project' s field effort and was used in planning school, 
year actWities. .This system had two principal limitations. The 
first was that it wSfe cumbersome to* use since categories suqh as the 
number of Sjfl}ools Involved "had to be'tallied by hand. The second was 
that participation in the project typically changed throughout the 
school year and, without repeating the sta^tewide survey, it was not 
possible to give accurate .totals at anyftime other than thebeginning 
' ' of . the' school year. Thrs led to some problems when, for example, 
decision piakers wanted a measure of the project's size during* the 
spring budget planning period or when the evaluators drew a study 
sample. /HFhe currenKdata base is maintained on a ^computer program 
that p*4duces readable reports and is updated three times ^ year^in 
the fall, in late winter, and shortly after the. close of the school 
year. The fall update provides a picture of the* project as soon as 
school enrollment has been completed. The late- winter update corrects 
. this to reflect subsequent changes in participation, and the summer 
^ update gives a current year-to-date measure of the schools that haVe 

met the COE. 
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Roles- 

There are three groups inyplved in the development of the SPUR 
Data Base and the production of SPUR project staff 

members act as djata coll6£tors, gathering the initial information and. 
correcting it as participation * changes. The Bureau ' of Evaluation 
maintains the Data Base aqd* produces the r^pbrts. The computer 

programming needed to produce the repbrts-Was deve^Jpted by the SDE 

' * . ' ^ * ■■■ . ■ ■ ' ' , ■ • » ' ' 

Bureau' of Management Infbrmation System^. . ' ^ ' 

How Developed ; ; ^ . ^ • • i' ^ 

The Louisiana Department of Education .SPUk staff had asked, 
during several years of the project's evaluation, for information 
about local participation in ISI^UR; Typically' this information was for 
decision-making audiences such as the Sta te Le gislature or the Board 
of Elementary and Secondary Educa^tion, and the information was needed 
•in a very^ short time. There was also a need for the evaluators to 
accurdtety identify participants for samples in activities such as 
surveys or testing. Thi^^as necessary because the SPUR field staff 
often acted as data collectti^ 'questionnaires or 

observing classrooms— and tightly controlled sampling hetlped to^^ard 
against the threat of bias. However, SPUR is ^ basically a field 
effort and . was not structured to ^naintain a constantly accurate 
Updated record of participants. SPUR and the evaluation unit a[gre^d 
/that' this task was one that could be appropriately allocated* to the 
evaluation. . \ 

The evaluation staff met with the St^te Si?UR staff members as 
well as representatives of the regional teams .to ^Identify the categor- 
ies of information that were needed, the'times at which this infor- 
mation would best fae updated, and the most efficient procedures for 
collecting and reporting the data. The decisipn was to collect the 
information shown in the sample data base report and to update it 
three times- la ye'ST^by having the. team members note changes on copies 
of the most recent report. This can be done, through the mail or at 
State and regipnal group meetings. The updated reports are submitted 
directly to .the Bureau of Evaluation. When ' the information is 
entered, a new "report is j^^rinted and sent to each regional office. 



The prograraiier from the Bureau of Management. Information Systems/ 
worked* with the .eyaluaJ;ors to develop a format that was as "user 
friendly" as possible. It /can - be maintained . by a person who has 
virtually no computer training and produces reports that .dVe easily 
read.by tfie SPUR field staff pijersonneJ. • 1* , 

In outline, the steps involved in developing , the SPUR Data Base 
.were as listed below: | 

1: /j^n information need was identified jointly by the project 
and the evaluators. . 

2. A potential system was drafted by' the evaluators and reviewed 
^ ' by computer programmers for practicality. • ' " 

^ 3. I The system was presented to the field staff for discussion 
. and revision/ • . 

4. The revised.^system was piloted.* , 

5. The sVstem was again revised. - ^ ^ ^ 

6. The syistem was implemented and maintained. V / " 

Training ' , . 

The . approximately 60 . SPUR field staff members act- as the data 
collectors fpr:' the system. After the system was establishe'd and a 
report produced from existing data, these persons were^trained in its' 
use during a half-daj^ session at a regularly scheduled statewide staff 
training meeting. The training ^covered the structure of the Data Base 
and the lises for which it was inilended. The latter step 'was 
particularly important because, the timeliness and accuracy of data 
collected by the. field jstaff members relate directly to their 
perception of how important the data are to SPUR. 



Information Use , , ' 

During this first' year that the SPUR Data Base ha^S been in 
operatioi^ the primary us6s have been (1) generating timely reports brt 
participation in SPUR and (2) producing accurateysampling lists fbr 
evaluation activities. Plans for next year incluoje expanding ^ these 
uses and adding a management function. As SPUR becomes .more accus- 
tomed to using it the Data Base should be useful in. schedul ing scTiool 

'•■ ' - ^-v ■„ ' ,. 24 ' 36 ; ; * ,.^J,. 
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COE reviews dnd determining audiences for statewide or ^ regional 



meetings, .conferences^ and irtservice training activities. 
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EXAMPLE 3: STtloENT ENGAGEMENT OBSERVATION SYSTEM 



What It Is 

This Is an observation system deyelbped for the evaluation- of the 
Louisiana SF^UR (Special Pi ati Upgradihg Reading) . Project. The system 
measures student time on task and the frequency of activities directed 
by the teacher during language arts. instruction* The' student engage- 
ment observation system is,, a slight modification of a methodology 
developed by Research for Better Schools, Inc.^ The teaching activi- 
ties were developed by the SPUR and the evaluat^^qn staffs!^ 



Student Engacjement Categories 



Management/Transition 
SociaVi^ing v 
Discipline 

Unoccupied/Observing 
Out of Room 
Engaged. 



Teacher Activity; Categories 

• Silent Reading, 

• Oral Reading 

• Writing/Composition - 

• Drill and Practice 

• Instruction 

• Discussion . 

• Test/Qyiz 

• Noninstriictional Management 




Th^ stuflent engagement observation system was initially developed a^ a 
part of a statew'ide evaluation for SPUR; it has 'since become a local 
program improvement, toolv that is no longer usfed in evaluation. The 
time frame in the system's evolution has been as follows: / 

■ r ■ • : 

• . 1980-'81,: Select/develop system; trg^fn observers; observe for 
evaluation. . ' / ' ^\ / / 

1981- 82: Re-train observers; use ^fpr evaluation. . 

1982- 83: Train local school system staff (overview). 

1983- 84: Train local school system staff (practice); imple- 
,ment system locally. ^ 



• 



Role 



The. Louisiana Department of Edup|}tion SPUR staff initially 



requested a measure" of studfent enga^etnent \in: its evaluation to 



determine this general range of . studept^ engagement rates in schools 



adopting the project and to* see if student engagement was related to 
reading achi.evi5meht*V Thfe SDE Bureau of Evaluation , researched existing 
methods to find one that was technically sound, congruent With SPUR's^ 
suggestions for instructional practice/ and simple enough that* it 
could be. learned fairly .ea^^^^ field staff • The State SPUR 

staff and the evalua tor tTien worked with these field SPUR Technical 
Assistants (TAs) to develop, the categories of teacher-directed activi- 
ties, that were added to^ the system. 

The- Bureau of EvalMatiori /trained the SDE SPUR Project Officers 
and field-based TAs to use the observation system. The JAs, in turn, 
trained teachers to complete their parts of the observations and 
carried out the observations. The data were returned to the Bureau of 
Evaluat-ion for analysis* ' , 

This process was repeated in the second y^ar. In the third year, 
of its use the observation system, was removed from the evaluation 
because SPUR and the evaluator agreed that sufficient baseline infdr^; 
mation had been collected." At t|iat time., training wasVoffere^ by the 
Bureau^ of EyaluatiiDn^ to. any SPUR centra-l office or ^school staff 
' . memBers^ho wgre interested in usin^ t*he procedure for program improve- 
ment. About jfbO people ^ participated . in these one-day Workshops. " 
Those who decided after receiving an overview to use the system then* 
worked with the SPUR TAs to learn precisely how to use it. In its 
fourth, year, the, system was used by local school systems to collect; 
information about student engagement that would, be helpful in improv- ^ 
ing local instructional programs. 
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How> Developed 

The observation "System^ was borrowed from what appeared to be the 
best co(npromfse between precision of observation and practicality of 
use. During its /dfevelopment , the Bureau of Evaluation drafted mater- 
ials, pa'ssed threm 'for review to the SDE SPUR staff, revised materials 
following .these 

astern, and-revisejl material ^gaia on l^iAj^sis of 
The-teacher-directed activities were deveTdpOT in -the 
^implified form of this revision prgcess was followed 
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in the secondr year in order to* iacorporate the suggestions of the TAs 
v<ho. ^had used the observation system; In outline, format,- the basic 

steps^were as follows; ' , 1 

• • ■ ' ','•■* 

; ' Eyaluatio^n negotiates needs wftb client "(•SPUR). 
2; .Research of potential systems. > 
3> r Review/discus§i with State Department of Education SPUR. 
4; Evaluation drafts 'total system with^ SPUR content input. 

5. V SDE SPUR staff reviews. ^ . , " / 

6. Evaluation revises. ^- * ' i ' 

7. SPUR Tech-nicaf Assistants review. :\ 
/ 8. Evaluation revises, i , / - > v. 

9. Evaluation trains observers. , % • • 

10. Oliservations carried out; data analyze'd: 

' 11.' pbservers critique existing mdterials^and procedures. ^ 

12. Evaluation re-trains observer?. \ 



Training , 

• The SPUR evaluator trained all of the TAs who would be conducting 
the observations a.s well as their supervisors during a two-day session 

:a few weeks before the* time scheduled for the observations. The 
training included an overview of the observation system/ the proce- 
dures for draWing and scheduling a sample ^pf observatiqns,, explanation 
of how to/complete both teachers' and observers' forms, and. practice 

"in using the observation with videotaped classroom examples. The 
training materials included explicit written explanations and instruct- 
ionsJ They also providecl the observers with materials they could use 
in training the teachers/to record/.their part of the observations. 

The traini^ was shortened apd simplified for lodal school system 
personnel whert^ ttfe- obgervation system changed ,f rom'an evaluation to a 
program, improvement^ component. In t)ie latter case the purpose of the 
training was to explain how the system worked so that , participants 
could determine whethe^^t would be usieful jind . practical for their 
needs. ^ For those wht)* wanted to pursue it ' further, the SPUR TAs' 

. provide^d later detailed training and supervised practice^in conducting 
the observations. ^ : * - * 



Information Use ■ 



. ..A number of ways weref tested to make the Informjation as O^eful as 
possible, and. the size, of /the groups participating in the training 
wheji it wag no longer a part of the SPUR evaluation j suggests thait- it 
was seen as useful. Evaluation reports on student engagement werje 
sent to* the local SPUR Director in each of the 64 participa|:ing public 
school systems as well as to the; SPUR Technical Assistance teams. 
One-page summaries of the results were also prepared 'afld distributed 
to a wider range of audiences, from superintendents of . schools to 
students in college courses. The findingls were presented at meetings 
such as the Louisiana Educational Research Association's annual 
conference. - * * ' . 

When, the observation system was changed from an evaluation to a 
' .pV^ogram improvemen|^ ^tool , two additional, steps were t^ken. , One, a 
background paper on student enga^gement research was prepared to give 
training partictpants a broader idea of the instri^ctional uses pf the 
observations. Two, efforts were made to coordinate the observation 
system with, a similar method developed by the Louisiana Chapter 1 
program in which a fiumber of teachers had been trained during the same 
year. 

The benefits of the qbservation system that have been reported 
anecdotally by its users include the following: 

1. It is a generic system^ appropriate for process evaluations 
' ^ in a variety of subject areas, and programs— math, social 

studies, Chapter, 1, etc. ' • 

1 ■ , ' ' ■* 

2.1 It forces the observer to look at students; the classroom 
observation systems' with ^whlch many had been familiaV 
concentrated almost exclusiveiy'on the teacher. ' 



3. The system is a* relatively nonthreaten^ng Way ' to get a^ 
teacher to examine her or his classroom. 

4. ' The Observation system can be used Nrs a pre-post measure to 
. see "if an instructional change has had an" effect upon* 
' student behavior. . • 

5. / Using the observation system has helped central ■ office 

supervisors and school principals to become ^more skiJled in 
observing teachers and more confident of their ability to do 
Jthis. : 
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